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EfFects-based  Assessment:  A  Research  Memo 

I.  Introduction,  Motivation  &  Audience.  Key  to  taking  an  effects-based  approach  to 
conflict  is  an  ability  to  assess  the  outcomes — or  effects — of  actions  taken  to  achieve  those  effects. 
Historically,  assessment  activities  focused  on  the  actions  themselves  and,  at  best,  on  progress 
towards  achieving  specific  objectives,  normally  tactical  objectives  at  component  levels  and 
operational  and  campaign  (or  theater)  objectives  at  the  Joint  Task  Force  or  Combatant  Command 
level.  The  motivation  for  this  research  memo  is  to  explore  one  way  effects-based  assessment 
(EBA)  can  be  modeled.  There  is  no  attempt  here  to  definitize  an  EBA  process  or  completely 
identify  necessary  information  requirements  for  an  EBA  model.  The  main  audience  for  this  memo 
is  technology  developers  who  might  require  some  process  and  information  requirements  to  begin 
designing  technologies  that  might  support  that  process.  One  would  hope  and  expect  through  this 
research  effort  to  identify  different  ways  and  means  of  conducting  EBA  all  with  the  same  end  state 
of  providing  decision  support  technologies  for  EBA. 

This  memo  proceeds  as  follows.  First  working  definitions  are  provided  then  EBA  is  explained 
from  the  current  state-of-practice  to  an  EBA  approach  to  assessment.  The  main  part  of  this  memo 
offers  one  approach  to  constructing  an  EBA  model  that  outlines  the  inputs,  activities,  and  expected 
outputs  from  the  model.  Finally  the  memo  ends  with  some  suggestions  for  future  research  into 
effects-based  assessment. 

II.  Working  Definitions.  The  following  working  definitions  are  used  in  this  memo.  None  of 
these  definitions  are  canonical.  Where  appropriate,  the  working  definition  also  provides  some 
explanation  on  how  it  differs  from  other  available  definitions. 

Model :  a  stylized  representation  of  reality  whose  level  of  detail  is  determined  by  the  required  use 
of  the  model.  The  structure  of  the  model  consists  of  the  rules,  generally  expressed  as  IF  -  THEN  - 
ELSE  statements  and  the  type  of  information  required  by  the  rule  statements.  The  actual 
information  is  separate  from  the  model.  Finally  BECAUSE  statements  are  added  to  the  rule 
statements  to  support  causal  linkage  assessment. 
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Assessment :  the  set  of  integrated  and  coherent  activities  that  determine  the  results,  and  the 
effectiveness  of  those  results,  from  the  execution  of  operations.  Memo  this  limits  assessment  to 
post-execution  and  ignores  the  critical  need  for  assessment  activities  that  predict  the  likely  results 
of  plans.  This  is  sometimes  referred  to  as  ‘plan  critique’.  This  narrow  definition  also  ignores  the 
‘planning  for  assessment’  activities  that  must  take  place  before  post-execution  assessment  can  take 
place.  Both  these  admittedly  crucial  activities  are  outside  the  scope  of  this  research  memo. 
Effects-based :  a  mindset  and  set  of  processes  that  focuses  on  outcomes,  results,  or  effects  of 
actions  (either  direct  or  indirect)  and  the  cause  of  those  results  rather  than  on  the  actions 
themselves. 

Effects-based  Assessment:  the  set  of  activities  that  determine  the  results  (outcomes  or  effects), 
and  the  effectiveness  of  those  results,  from  the  execution  of  operations  beyond  the  immediate 
results  of  those  operations.1  The  March  2004  CAF  (Combat  Air  Forces)  White  Paper,  “Effects- 
Based  Assessment:  Closing  the  Loop”  does  not  expressly  define  EBA.  It  views  EBA  as  the 
combination  of  tactical  assessment,  component  effects  assessment,  and  operational  effects 
assessment.  These  definitions  appear  to  add  more  detail  to  the  working  definition  used  here  but  do 
not  conflict  in  any  way  with  the  working  definition  used  here.  The  March  2004  Draft  AFDC  (Air 
Force  Doctrine  Center)  White  Paper,  “The  Effects-based  Approach  to  Conflict”  states  that 
assessment  consists  of  all  efforts  to  evaluate  performance  and  progress.  This  definition  appears 
much  more  general  than  the  working  definition  used  in  this  memo  but  again  does  not  appear  to 
conflict  in  any  way  with  the  working  definition  used  here.  It  is  worth  noting,  however,  that  both 
the  CAF  and  AFDC  white  papers  emphasize  that  post-execution  assessment  (the  focus  of  this 
research  memo)  is  not  done  or  should  not  be  viewed  in  isolation  from  planning,  execution  or  plan 
critique  activities.  This  meshing  of  planning,  execution,  and  assessment  is  a  constant  theme  in  this 
research  memo  even  though  the  focus  of  the  memo  is  on  a  model  for  post-execution  effects-based 
assessment. 

III.  Explanation.  This  section  places  EBA  within  the  context  of  current  combat  assessment 
practices.  The  main  point  here  is  that  EBA  does  not  completely  replace  current  practice  but  rather 

1  The  Joint  and  canonical  concept  is  called  combat  assessment  and  is  defined  as  “The  determination  of  the  overall 
effectiveness  of  force  employment  during  military  operations.”  See  JP  1-02.  The  CAF  and  AFDC  definitions  are  used 
here  for  comparison  because  they  both  seek  to  address  issues  of  effects  assessment  whereas  Joint  doctrine,  at  least  so 
far,  is  silent  on  this  question. 
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builds  upon  it  to  produce  a  more  complete  assessment  picture.  In  this,  EBA  is  the  same  as  effects- 
based  planning  (EBP)  which  is  best  viewed  as  an  extension  to,  and  modification  of,  the  current 
Joint  Estimate  Process  rather  than  a  completely  new  set  of  processes. 

a.  Current  Practice:  Combat  Assessment.  According  to  Joint  Publication  3-60,  Joint  Doctrine 
for  Targeting,  Combat  Assessment  (CA)  is  the  sixth  and  final  phase  of  the  Joint  Targeting  Cycle. 
It  consists  of  two  parts:  battle  damage  assessment  (BDA)  and  munitions  effectiveness  assessment. 
The  result  of  these  two  processes  is  a  reattack  recommendation  or  a  requirement  for  future  target 
development.  The  main  interest  for  this  research  memo  is  the  BDA  processes.  Overall,  the  CA 
process  is  designed  to  determine  whether  the  Joint  Force  Commander’s  desired  effects  on  the 
adversary,  as  envisioned  in  the  campaign  plan,  are  being  achieved.  BDA  supports  this  goal  by 
taking  a  three  level  approach  from  the  micro  to  the  macro  view  of  the  adversary.  These  levels  are 
described  in  more  detail  below.  The  last  point  about  current  practice  to  emphasize  is  the  close 
connection  between  CA  and  planning.  During  planning  measures  of  effectiveness  (MOE)  and 
performance  (MOP)  are  developed  and  these  are  used  during  CA  in  determining  whether  the 
Commander’s  goals  are  being  achieved  or  not. 

b.  Effects-based  Assessment.  EBA  builds  on  these  two  CA  principles:  BDA  reports  that  examine 
the  effects  on  an  adversary  across  multiple  levels  and  the  use  of  measures  to  ascertain  progress 
towards  attaining  the  Commander’s  desired  effects.  Fundamentally  EBA  adds  to  these  principles 
one  that  deals  with  explaining  the  linkage  between  actions  and  effects.  Furthermore,  besides 
classic  MOE  and  MOP  information,  EBA  specifies  beforehand  the  evidence  of  actions,  causal 
linkages,  and  outcomes  (or  effects).  These  pre-determined  bits  of  information  are  collectively 
referred  to  as  ‘Indicators’.  The  basic  EBA  process  is  by  marrying  the  Indicators,  MOE  and 
MOP — that  is,  the  evidence,  measures  and  criteria — with  the  BDA  reports  that  provide  the  impact 
on  the  adversary  from  the  target  through  the  target  system  level,  a  more  complete  picture  of  the 
impact  of  military  actions  on  the  adversary  is  provided.  More  detail  on  this  process  is  provided 
below.  Readers  will  memo  the  term  ‘fusion’  is  avoided  here.  While  it  might  be  technically  correct, 
fusion  has  a  specific  technical  meaning  that  is  much  narrower  than  the  idea  advanced  here.  Fusion 
is  definitely  required  but  more  than  fusion  is  needed  to  fulfill  the  concept  of  EBA  advanced  in  this 
research  memo. 

c.  Summary  of  Major  Differences.  Because  effects-based  plans  are  richer  in  content  than  classic 
plans,  there  is  correspondingly  more  information  required  to  accomplish  effects-based  assessment. 
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This  increased  information  requirement  cuts  both  ways.  On  the  one  hand,  it  is  often  commented 
that  commanders  are  ‘data  rich  but  information  poor’.  In  such  an  environment,  adding  additional 
information  requirements  seems  likely  to  make  a  bad  situation  worse.  On  the  other  hand,  because 
more  data  is  presumably  available  than  previously,  at  least  the  potential  exists  that  more,  and 
perhaps  better,  information  could  be  made  available  to  a  commander.  The  second  difference 
between  classic  CA  and  EBA  is  more  fundamental.  The  traditional  approach  to  CA  has  been 
called  ‘rollup’  because  the  findings  at  lower  levels,  such  as  the  MOP  and  physical  effect  levels,  are 
‘rolled  up’  to  higher  and  higher  levels  based  solely  upon  what  was  planned.2  This  seems  premised 
on  the  principle  that  the  planners  got  the  decomposition  from  objectives  to  tasks  correct  so  that  if 
the  tasks  were  successfully  accomplished  then  ipso  facto  the  objectives  were  accomplished. 
Unfortunately  the  historical  record  does  not  support  such  optimism.  EBA  attempts  to  remedy  this 
defect  by  specifically  identifying  the  evidence  that  would  support  the  causal  chain  from  action  to 
effect  to  specifically  include  looking  for  evidence  to  ascertain  the  correctness  of  the  anticipated 
causal  linkage  (or  mechanism)  between  action  and  effect. 


00  =  Operational  Objective;  TO  =  Tactical  Objective 


2  Unfortunately,  this  approach  is  also  advocated  in  the  CAF  White  Paper  on  EBA.  See  p.  7. 
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IV.  EBA  Model.  This  section  provides  the  EBA  model  in  the  form  of  a  matrix  (called  EBAM). 

■5 

See  Figure.  The  genesis  of  this  matrix  is  the  Target  Value  Matrix  described  in  Joint  doctrine.  This 
form  was  chosen  because  it  allows  depiction  of  the  EBA  elements,  described  briefly  above  and 
more  fully  here,  and  their  relationships.  The  matrix  is  described  in  terms  of  inputs,  activities,  and 
outputs.  This  allows  the  matrix  to  be  transformed  into  IDEF  ICOM  products3 4  if  desired.  The 
architecture  could  then  be  expanded  by  adding  applications  or  technologies  that  would  support  the 
resulting  activity  models. 

a.  Inputs:  An  effects-based  plan.  The  input  into  EBAM  is  the  causal  model  built  during  effects- 
based  planning.  The  details  of  that  model  are: 

Effects.  This  is  the  outcome  anticipated  from  the  action  (direct  Effect)  or  the  result  from 
previous  actions  (indirect  Effect)  or  some  combination  of  actions  and  previous  results 
(cumulative  Effect).  The  action  or  previous  result  (or  some  combination)  comprise  the  IF 
while  the  direct,  indirect,  or  cumulative  Effect  comprise  the  THEN  forming  the  basic  IF  - 
THEN  relationship.  Example:  IF  the  bomb  explodes  where  targeted,  THEN  the  bridge  is 
destroyed.  The  sentence  is  generally  written,  however,  as  “The  bridge  is  destroyed”  as  a 
statement  of  Effect. 

Objects  of  Effects:  Role  of  System-of-Systems  analysis  (SoSA).  Systems  analysis  (also 
called  network  analysis,  target  systems  analysis,  or  system-of-systems  analysis)  is  crucial 
for  an  effects-based  approach  to  planning,  execution,  and  assessment.  It  is  how  indirect  and 
cumulative  Effects  are  predicted  during  planning  and  how  outcomes  are  assessed  beyond 
first-order  physical  effects.  SoSA  provides  the  objects  for  the  desired  effect.  In  the 
example  above,  ‘bridge’  was  the  object  of  the  effect.  According  to  Joint  doctrine,  a  target 
system  is  one  where  targets  are  functionally  related  and  that  relationship  is  such  that  an 
effect  on  one  target  in  the  system  will  produce  some  effect  on  the  system  as  an  entity. 
Hence  target  systems  can  first,  be  functionally  decomposed.  Secondly,  target  systems 
exhibit  dependency  both  within  themselves  and  between  themselves  and  other  target 
systems.  This  is  referred  to  in  Joint  doctrine  and  “intra-dependency”  (within  themselves) 
and  “inter-dependency”  (between  themselves  and  other  target  systems).  Research  also 
shows  there  are  causal  relationships  both  within  and  between  target  systems.  That  is,  a 

3  See  JP  2-01.3. 

4  Inputs,  Controls,  Outputs,  Mechanisms. 
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change  in  state  of  one  element  causes  a  change  in  state  of  one  or  more  other  elements.  In 
summary,  target  systems  or  SoSA  must  show  the  elements  of  a  system  (generally  called 
‘nodes’)  and  three  relationships  (generally  called  ‘links’):  functional,  dependent,  and 
causal.5 

Causal  Linkages  or  Mechanisms.  Causal  linkages  are  explanations  that  state  why  the 
actions  (or  previous  actions  or  indirect  Effects)  will  result  in  the  anticipated  outcome. 
Generally  these  are  expressed  in  a  probabilistic  manner  with  the  probabilities 
proportionately  decreasing  from  the  first-order  physical  effect  level  to  the  n-order,  mainly 
behavioral,  effect  level  such  as  that  associated  with  strategic  outcomes.  Causal  linkages 
provide  the  BECAUSE  element  in  IF  -  THEN  statements. 

Timing .  While  not  technically  an  element  in  an  effects-based  plan  (any  plan  requires  some 
idea  of  time),  timing  plays  an  especially  crucial  role  in  EBP  and  hence  in  EBA.  This  is 
because  of  delay  and  persistence.  Delay  is  the  time  from  action  to  effect.  This  can  be 
straightforward  when  dealing  with  direct  actions  and  first-order  physical  effects  but  can 
become  quite  complicated  when  dealing  with  cumulative  effects.  The  impact  on  EBA  is 
clear.  Information  may  be  available  from  near  instantaneous  to  several  days;  however,  the 
desired  effect  may  not  be  present  (or,  and  this  is  a  different  timing  calculation,  observable) 
within  those  timelines.  This  could  lead  to  a  false  negative  assessment.  Persistence  is  the 
time  an  effect  lasts  without  further  input.  Again,  the  more  complex  the  effect  sought,  the 
more  complex  the  calculation  required. 

Indicators.  As  defined  above,  Indicators  provide  the  evidence  planners  anticipate  will 
show  the  various  elements  in  the  causal  model.  There  can  be  various  types  of  evidence 
sought  depending  on  the  nature  of  the  element.  This  variety  provides  collection  managers 
more  options  in  assembling  the  competing  tasks  for  their  scarce  intelligence  gathering 
assets.  It  is  preferable  that  the  evidence  sought  be  of  the  type  that  can  be  observed  but  that 
is  not  a  firm  requirement.  There  are  Indicators  of  each  element  of  the  causal  model  from 
the  tactical  task  level  up  through  the  campaign  objectives  and  effect  level. 


5  A  useful  model  for  this  process  can  be  found  in  Joe  Strange,  “Center-of-Gravity  &  Critical  Vulnerabilities,” 
Perspectives  on  Warfighting,  Number  4,  Second  Edition,  Marine  Corps  University,  2002  (1996). 
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b.  Process:  Marrying  Indicator  data  with  BDA.  This  subsection  title  sums  the  basic  process  of 
EBA.  Indicators  were  discussed  above.  Now  the  attention  turns  to  the  other  sources  of  information 
used  to  conduct  EBA  and  closes  with  some  techniques  that  can  be  employed. 

1)  Levels  of  BDA.  The  main  information  sources  other  than  Indicators  are  Mission  Reports 
(MISREPS)  and  battle  damage  assessment  reports  (BDAREPS).  While  MISREPS  are 
useful  in  conducting  operational  level  assessments,  e.g.,  component  effects  assessment  and 
operational  effects  assessment,  they  are  most  useful  in  tactical  effects  and  events 
assessment  and  are  not  further  discussed  here.  There  are  three  phases  of  BDA  and  each 
produces  a  corresponding  level  of  battle  damage  assessment  provided  via  a  specific 
BDAREP.  The  first  phase  is  Physical  Damage  Assessment  (PDA).  It  is  an  estimate  of  the 
quantitative  extent  of  physical  damage  done  to  a  specific  target  (or  node  in  a  target 
system).  According  to  JP  2-01.1,  “Some  representative  sources  for  data  necessary  to  make 
a  physical  damage  assessment  include  the  air  tasking  order  (ATO)  or  master  air  attack 
plan,  MISREPs,  aircraft  cockpit  video  (ACV),  weapon  system  video  (WSV),  visual/verbal 
reports  from  ground  spotters  or  combat  troops,  controllers  and  observers,  artillery  target 
surveillance  reports,  SIGINT,  HUMINT,  IMINT,  MASINT,  and  open-source  intelligence 
(OSINT).”  The  second  phase  is  Functional  Damage  Assessment  (FDA).  It  is  an  estimate  of 
the  target’s  ability  to  perform  its  intended  function.  FDA  is  inferred  from  PDA  and  other 
sources  of  information.  FDA  also  includes  some  estimate  of  how  long  the  target  required 
to  recuperate  from  its  current  status  to  its  pre-attack  functional  capability.  The  last  phase  of 
BDA  is  target  system  assessment.  This  provides  a  broad  evaluation  of  the  effectiveness  of 
the  military  force  applied  against  the  target  system  in  relationship  to  the  effect  desired. 

2)  Marrying  information:  Quantification  &  Qualification.  As  shown,  Indicator,  BDAREP 
and  other  sources  of  information  comes  in  a  variety  of  media  and  formats  from  video  to 
freeform  text  to  numbers.  Combining  these  is  not  a  trivial  problem.  Current  practice  is  a 
manual  process  supported,  if  at  all,  by  MS  Office  applications.  The  technique  offered  here 
uses  information  extraction  techniques,  coding  techniques,  and  an  ontology  base.  The 
process  starts  with  taking  the  extracted  information  and  matching  it  against  the  ontology 
base  to  gain  a  common  semantic,  machine  readable,  formal  language  rendition.  This  is 
codified  to  allow  statistical  manipulation  of  the  information  to  support  inferential 
reasoning.  The  resulting  data  is  combined  with  the  already-provided  quantified  data  then 
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analyzed  using  standard  inferential  statistical  reasoning  techniques.  The  challenge  to  this 
technique,  of  course,  is  the  ontology  base.  While  difficult  in  any  case,  providing  ontology 
for  behavioral  effects  is  especially  daunting.  While  efforts  are  underway,  they  are  far  from 
complete.  Once  the  coding  and  statistical  manipulation  is  complete,  the  information  is 
plugged  back  into  the  causal  model  where  tools  used  for  plan  critique  can  now  be  used  for 
post-execution  assessment. 

c.  Outputs:  Status,  Problems  &  Opportunities.  Presenting  the  analyzed  information  is  normally 
done  via  ‘stoplight  charts’  where  red,  yellow,  green,  and  blue  circles  are  generally  displayed  in  a 
time-series  to  show  past  accomplishment,  present  status,  and  predicted  future  state  at  each 
objective  and  effect  level.  The  top  figure  below  shows  an  example  used  during  JEFX04.  The 
bottom  figure  below  shows  an  example  of  what  was  used  by  CDRUSCENTCOM  during 
Operation  IRAQI  FREEDOM. 

The  use  of  inferential  statistical  methods  also  allows  presenting  confidence  levels  with  the 
assessment.  Finally,  drilldown  techniques  allow  assessors  to  examine  in  detail  where  problems 
may  have  arisen  in  the  plan.  Less  often  those  techniques  provide  any  suggestions  on  how  to 
remedy  those  problems.  Even  rarer  are  techniques  that  use  operational-level  assessment  to  identify 
opportunities.  AFRL's  causal  analysis  tool  (CAT)  is  an  exception  to  this. 
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V.  Areas  for  Further  Research.  This  quick  survey  on  one  approach  to  EBA  points  out  three 
areas  needing  further  investigation.  These  areas  are  not  the  only  ones  needing  further  investigation 
nor  do  they  solely  need  research  only  within  the  context  of  EBA.  Rather  these  three  were  chosen 
as  essential  to  completing  the  operational  requirements  that  derive  from  the  EBA  approach  offered 
here.  The  first  deals  with  system-of-systems  analysis.  The  main  issue  here  is  finding  ways  for 
commensuration  of  disparate  engineering  and  logical  models  such  that  inter-dependency  and 
causal  relationships  can  be  examined  together  and  predicted  outcomes  identified.  The  second  area 
is  covered  under  the  broad  title  of  fusion  of  data  not  only  from  disparate  data  sources  but  from 
disparate  media  types  and  with  a  variety  of  pedigrees,  levels  of  ambiguity,  and  missing  elements. 
Finally,  further  research  is  needed  into  the  processes  of  bringing  all  the  various  elements  of  EBA 
together  such  that  they  are  integrated  and  coherent,  both  internally  (as  part  of  EBA)  and  as  one 
element  in  an  effects-based  approach  to  planning,  executing,  and  assessing  operational-level 
operations. 


10 


Glossary 


ACV 

Aircraft  cockpit  video 

AFDC 

Air  Force  Doctrine  Center 

AFRL 

Air  Force  Research  Laboratory 

ATO 

Air  tasking  order 

BDA 

Battle  damage  assessment 

CA 

Combat  assessment 

CAF 

Combat  air  forces 

CAT 

Causal  analysis  tool 

CFACC 

Coalition  Forces  Air  Component  Commander 

EBA 

Effects-based  assessment 

FDA 

Functional  damage  assessment 

HUMIT 

Human  intelligence 

IMINT 

Image  intelligence 
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Information  operations 

JEFX 

Joint  Expeditionary  Force  Experiment 

JP 

Joint  publication 

MASINT 

Measurements  and  signatures  intelligence 

MISREP 

Mission  report 

MOE 

Measure  of  effectiveness 

OSINT 

Open  source  intelligence 

PDA 

Physical  damage  assessment 

REP 

Report 

SIGNINT 

Signals  intelligence 

WMD 

Weapons  of  mass  destruction 

wsv 

weapon  system  video 
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